Skip to content

neon: sync catalog with live AI Gateway probe (add 12 models, remove gpt-5-5)#3019

Merged
rekram1-node merged 5 commits into
anomalyco:devfrom
andrelandgraf:neon-catalog-sync-2026-07
Jul 9, 2026
Merged

neon: sync catalog with live AI Gateway probe (add 12 models, remove gpt-5-5)#3019
rekram1-node merged 5 commits into
anomalyco:devfrom
andrelandgraf:neon-catalog-sync-2026-07

Conversation

@andrelandgraf

@andrelandgraf andrelandgraf commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Updates the Neon provider to match what the Neon AI Gateway actually serves today. Neon's gateway is powered by Databricks Foundation Model APIs, so I took the current Databricks supported-models list (docs updated 2026-07-02) as the candidate set and probed every model live against a real Neon AI Gateway branch (us-east-2), then live-checked capabilities. The current Neon entries here were missing 11 working models and listed one (gpt-5-5) that the gateway rejects.

Neon catalog id = Databricks pay-per-token endpoint name minus the databricks- prefix (e.g. claude-sonnet-4-6 → underlying global.anthropic.claude-sonnet-4-6).

Added (12) — all verified working via live calls

Proprietary (pass-through provider list pricing), via base_model:

  • gpt-5-3-codex, gpt-5-2-codex, gpt-5-1-codex-max, gpt-5-1-codex-mini (OpenAI Responses API)
  • claude-opus-4-8
  • gemini-3-5-flash

Open-weight (priced from Databricks pay-per-token DBU rate × $0.070/DBU):

  • llama-4-maverick — $0.50/$1.50
  • meta-llama-3-3-70b-instruct — $0.50/$1.50
  • meta-llama-3-1-8b-instruct — $0.15/$0.45
  • qwen35-122b-a10b — $0.22/$2.20
  • qwen3-next-80b-a3b-instruct — $0.15/$1.20
  • gemma-3-12b — $0.15/$0.50

Removed (1)

  • gpt-5-5 — the gateway returns 400 unknown model "gpt-5-5" (any prefix / dialect). Not currently served by Neon.

How this was verified

For each candidate I sent real requests to the gateway and recorded the result:

  • works? — minimal completion (chat-completions dialect, or the OpenAI Responses dialect for the codex models). All 12 above returned 200.
  • image input (attachment) — sent a generated 32×32 PNG data-URI + "what color?". 200 ⇒ supported; 400 does not support image ⇒ not.
  • tool calling (tool_call) — sent a get_weather function tool and checked for an emitted tool call. All 12 support tool calling.

Capability corrections applied so the entries match the gateway (not just the canonical base model):

  • qwen35-122b-a10b: canonical Qwen3.5 is multimodal, but Databricks/Neon serve it text-only (live: no image input). Overridden to attachment = false, modalities.input = ["text"], and limit.output = 8000 per the Databricks doc.
  • meta-llama-3-3-70b-instruct: overridden to attachment = false (text-only on the gateway; live-confirmed).

Notes / open questions for maintainers

  • The databricks:generate script currently IGNOREs llama / qwen / gemma prefixes, so the Databricks provider omits them. These do work on Neon, so I've added them here with Databricks DBU-derived pricing — happy to drop them if you'd prefer Neon to mirror that ignore-list.
  • Open-weight pricing uses the published Databricks pay-per-token DBU rates × $0.070/DBU (the rate that reproduces the headline GPT-OSS-120B $0.15/$0.60). The pre-existing gpt-oss-120b/gpt-oss-20b Neon entries appear to use OpenRouter pricing instead (e.g. gpt-oss-120b = $0.072/$0.28 rather than Databricks' $0.15/$0.60) — left unchanged here, but flagging the inconsistency.
  • Also verified not available on Neon and intentionally not added: claude-sonnet-5, claude-fable-5, meta-llama-3-1-405b-instruct (unknown model), the Gemini image-output models, and all embedding models (the gateway exposes no /embeddings route).

Test plan

  • bun run validate-equivalent: parsed each new TOML, resolved base_model merges, and confirmed required fields (name, attachment, reasoning, tool_call, open_weights, release_date, last_updated, cost.{input,output}, limit.{context,output}, modalities.{input,output}). All 36 Neon models valid.
  • CI schema validation (GitHub Action).

Update — reasoning controls & fast mode corrected (live re-verified)

A maintainer asked (via DM) whether Neon actually serves Opus fast mode and how the qwen35 reasoning controls work. I re-probed the live gateway (us-east-2) and corrected two entries in this PR:

claude-opus-4-8

  • Removed [experimental.modes.fast]. The gateway rejects fast mode on every request shape:
    • speed:"fast" + anthropic-beta: fast-mode-2026-02-01 on /ai-gateway/anthropic/v1/messages400 "speed: Extra inputs are not permitted"
    • beta header only → 400 "invalid beta flag"
    • same via /ai-gateway/mlflow/v1/chat/completions400 "speed: Extra inputs are not permitted"
    • (Opus 4.7 and 4.6 behave identically — fast mode is not exposed by Neon at all.)
  • reasoning_options: toggle + budget_tokenstoggle + effort. thinking.type:"enabled" + budget_tokens returns 400 '"thinking.type.enabled" is not supported for this model. Use "thinking.type.adaptive" and "output_config.effort" to control thinking behavior.' The gateway accepts thinking.type:"disabled" (toggle off, 200) and thinking.type:"adaptive" + output_config.effort. Its own validation error enumerates the accepted effort values: low, medium, high, xhigh, max (negative control effort:"banana"400 "unknown variant 'banana', expected one of 'low','medium','high','xhigh','max'"). none/minimal/default are rejected.

qwen35-122b-a10b

  • reasoning_options: toggle + budget_tokenseffort (none/low/medium/high). On the mlflow chat-completions route, thinking, enable_thinking, and any reasoning-budget field all return 400 "unknown field". The real control is reasoning_effort; the gateway's validation error enumerates the accepted values (parameter "reasoning_effort" must be "none", "low", "medium" or "high"). reasoning_effort:"none" returns a plain, non-reasoning completion (reasoning off); the other values return type:"reasoning" blocks. There is no separate budget or enable_thinking toggle — none is the off state.

All checks were minimal live requests against a real Neon AI Gateway branch; no budget_tokens control exists for either model on Neon today.

claude-opus-4-7 and claude-opus-4-6 (pre-existing entries, corrected here too)

Both carried the same [experimental.modes.fast] block, and the gateway 400s fast mode on both (speed: Extra inputs are not permitted) — removed from both.

Reasoning differs between the two:

  • claude-opus-4-7 behaves like 4-8: thinking.type:"enabled" + budget_tokens400 (must use adaptive + output_config.effort); disabled toggles off; effort enum low/medium/high/xhigh/max (verified, incl. effort:"max" producing thinking tokens). → reasoning_options changed from toggle + budget_tokens to toggle + effort.
  • claude-opus-4-6 still supports the classic API: thinking.type:"enabled" + budget_tokens=1024 returns thinking blocks (200), and disabled toggles off. → reasoning_options left unchanged (toggle + budget_tokens is correct); only the unsupported fast mode block was removed.

Verified every Databricks Foundation Model API endpoint against a live Neon
AI Gateway branch (us-east-2). Adds 12 models confirmed working (with
live-checked image-input + tool-calling capabilities) and removes gpt-5-5,
which the gateway rejects as an unknown model.
The two inline models (no base_model to inherit from) were missing the
schema-required `description` field, failing CI validation.
Qwen3.5 122B inherits reasoning=true, so the schema requires reasoning_options.
Mirrors the canonical alibaba entry (toggle + budget_tokens).
Live-verified against the Neon AI Gateway (us-east-2):

- claude-opus-4-8: the gateway rejects `speed:"fast"` +
  `anthropic-beta: fast-mode-2026-02-01` with 400 ("speed: Extra inputs
  are not permitted" / "invalid beta flag") on both the anthropic and
  mlflow routes, so the model does not serve fast mode -> remove
  [experimental.modes.fast]. Reasoning is not budget_tokens-based:
  `thinking.type:"enabled"` + budget_tokens returns 400 ("use
  thinking.type.adaptive and output_config.effort"). The gateway accepts
  `thinking.type:"disabled"` (toggle off) and adaptive thinking with
  output_config.effort; its own validation error enumerates the accepted
  values (low/medium/high/xhigh/max) -> switch to toggle + effort.

- qwen35-122b-a10b: `thinking`, `enable_thinking`, and reasoning budget
  fields all return 400 ("unknown field"). The real control is
  `reasoning_effort`; the gateway's validation error enumerates
  none/low/medium/high, and `reasoning_effort:"none"` returns a plain
  (non-reasoning) completion -> switch to effort (none/low/medium/high).
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Review: neon catalog sync (PR #3019)

Verdict: No hard blockers — approve with minor follow-ups.

Blocker checklist (all pass)

  • base_model usage: All 10 factored entries point at existing models/ metadata (anthropic/claude-opus-4-8, google/gemini-3.5-flash, openai/gpt-5.{1-codex-max,1-codex-mini,2-codex,3-codex}, meta/llama-4-maverick-17b-instruct, meta/llama-3.3-70b-instruct, alibaba/qwen3-next-80b-a3b-instruct, alibaba/qwen3.5-122b-a10b). The two inline definitions (gemma-3-12b, meta-llama-3-1-8b-instruct) have no matching models/ entry, so inline is compliant.
  • reasoning_options: Every reasoning = true model declares options; every reasoning = false model correctly omits them (schema forbids options when reasoning is false).
  • Pricing/capability fidelity: The 6 proprietary entries match their canonical provider TOMLs exactly — codex effort values and per-1M pricing match providers/openai/models/gpt-5.*codex*.toml verbatim, and gemini-3-5-flash matches providers/google/models/gemini-3.5-flash.toml. Strong signal the pass-through pricing was sourced carefully.
  • Schema shape: Inline files carry all required fields (incl. description added in patch 2); no stray fields that would trip .strict(). Commits 2–3 show the author fixed the validation failures (description, reasoning_options for the reasoning qwen) before finalizing.
  • Live-probe evidence for the two non-canonical reasoning_options (4-8 toggle+effort, qwen3.5 effort) meets the audit-reasoning-options skill's "reproducible request + negative control" tier — the commit messages record the exact 400 responses that rejected budget_tokens/thinking/enable_thinking.

Non-blocking recommendations

  1. Cite the Databricks pay-per-token pricing source. The 6 open-weight prices are derived from "DBU rate × $0.070/DBU" but the PR body links no Databricks pricing page. The proprietary prices are self-verifying (they mirror canonical providers); the DBU-derived ones are not. Per the audit skill, add an Evidence section to the PR body with a link to the Databricks pay-per-token/DBU page and state what it proves. Also state the explicit toggle sentence for 4-8: "claude-opus-4-8 toggles reasoning with thinking.type set to adaptive or disabled."

  2. Follow-up: re-probe the existing Neon Claude siblings. claude-opus-4-7, -4-6, -4-5, and claude-sonnet-4-6 still declare budget_tokens + [experimental.modes.fast] (speed:"fast" / anthropic-beta: fast-mode-2026-02-01). This PR live-verified that 4-8 rejects both. If the gateway rejects them for 4-8, the older entries are likely unverified and may also be wrong — worth a separate probe so the Neon Claude family is consistent. (Not a blocker for this PR, which only touches 4-8.)

  3. Optional: a Neon sync module. If the gateway exposes a /v1/models-style list, wiring it into bun models:sync would keep the catalog current without manual probes. Given pricing still needs DBU derivation and capabilities need probing, hand-authoring remains defensible — only worth it if the endpoint is rich enough to be authoritative for deletions/IDs.

Live-verified against the Neon AI Gateway (us-east-2), same as opus-4-8:

- claude-opus-4-7: fast mode 400s ("speed: Extra inputs are not
  permitted"), and `thinking.type:"enabled"` + budget_tokens 400s ("use
  thinking.type.adaptive and output_config.effort"). `thinking.type:
  "disabled"` (toggle off) and adaptive + output_config.effort both work;
  the gateway's own validation error enumerates the accepted effort
  values (low/medium/high/xhigh/max). -> remove [experimental.modes.fast],
  switch reasoning_options to toggle + effort.

- claude-opus-4-6: fast mode 400s the same way -> remove
  [experimental.modes.fast]. Its reasoning_options are left unchanged:
  `thinking.type:"enabled"` + budget_tokens is accepted here (returns
  thinking blocks), so toggle + budget_tokens is correct for 4-6.
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Let me verify the Anthropic canonical opus-4-6/4-7 reasoning_options to assess Neon's reasoning-option choices, and confirm the base metadata reasoning flags for those models.

@rekram1-node rekram1-node merged commit 010385c into anomalyco:dev Jul 9, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants